test: Add E2E unit tests for CLP connector. - #54
Merged
anlowee merged 19 commits intoSep 3, 2025
Merged
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR updates enables the end-to-end testing functionality to CLP connector. The end-to-end testing feature is bundled for all native connectors such as Hive, Iceberge, and TPCH, so it is good for CLP to support it as well. Another important benefit is that it can save a lot of time setup the end-to-end testing environment and also ensure the we don't break functionality for each PR.
The first change is instead of
ClpMetadataDbSetUpwe implemented anotherClpMockMetadataDatabase. This class provides a builder style way to setup the metadata database. For consistency, besidesArchivesTableRowswe also definedColumnMetadataTableRowsandDatasetsTableRowsto represent the three tables that will be used by CLP connector, andClpMockMetadataDatabasehas some methods to register the necessary information by the data whose types are these three classes.The second change is in the
presto-native-executionmodule. Similar toTestPrestoNativeHiveGeneralQueriesandTestPrestoNativeIcebergGeneralQueries, we implementedTestPrestoNativeClpGeneralQueries. The major difference between the last one and the former two is that the entire framework is designed for those have both Java worker implementation and Pretissimo implementation, so they can use a Java coordinator and a Java worker (expected query runner) to compare with the Java coordinator and a Pretissimo worker (actual query runner) to do the unit testing. Also the native worker creation is hardcoded to createhiveconnector, so we also create a functiongetExternalClpWorkerLauncherto create our own worker.This PR only adds a
select *query for sanity check. Later, we should integrate theTestClpMetadata,TestClpSplitandTestClpQueryBase. Eventually, we should get rid ofClpMetadataDbSetUp. The tests related to the metadata database should be all integrated into the end-to-end testing framework. The end-to-end testing framework should also cover some basic queries to validate the functionality. Forpresto-clptests, we should only keep those testing without the need of metadata database, for example the pushdown generation etc.Checklist
breaking change.
Validation performed
Passed the CI (the unit test class will be automatically added into CI):

Summary by CodeRabbit
Tests
Chores
Notes